CLIP: Cheap Lipschitz Training of Neural Networks

Despite the large success of deep neural networks (DNN) in recent years, most still lack mathematical guarantees terms stability. For instance, DNNs are vulnerable to small or even imperceptible input perturbations, so called adversarial examples, that can cause false predictions. This instability have severe consequences applications which influence health and safety humans, e.g., biomedical imaging autonomous driving. While bounding Lipschitz constant a network improves stability, methods rely on restricting constants each layer gives poor bound for actual constant. In this paper we investigate variational regularization method named CLIP controlling network, easily be integrated into training procedure. We mathematically analyze proposed model, particular discussing impact chosen parameter output network. Finally, numerically evaluate our both nonlinear regression problem MNIST Fashion-MNIST classification databases, compare results with weight approach.